Electronic apparatus and control method thereof

ABSTRACT

Provided herein is control method of an electronic apparatus, comprising: identifying a line region from a driving-related image data of a vehicle; generating a line information corresponding to a lane where the vehicle is located from an image data of the identified line region portion; generating a position information of a lane where the vehicle is located, using at least one of the generated line information and the lane information of a road where the vehicle is located; and performing a driving-related guide of the vehicle using the generated lane position information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. nonprovisionalapplication Ser. No. 15/605,485 filed on May 25, 2017, which is in turna continuation application of U.S. nonprovisional application Ser. No.14/742,004 filed on Jun. 17, 2015, which claims priority from KoreanPatent Application Nos. 10-2014-0073727, filed on Jun. 17, 2014, and10-2014-0154597, filed on Nov. 7, 2014, in the Korean IntellectualProperty Office, the disclosures of which are incorporated herein byreference in their entirety.

BACKGROUND OF THE INVENTION 1. Field Of The Invention

The present invention relates to an electronic apparatus and a controlmethod thereof, and in particular to an electronic apparatus and acontrol method thereof which can perform a position recognition of alane where a vehicle is located, and a driving-related guide accordingto the same.

2. Description Of The Related Art

The most important thing when a user drives a vehicle is a safe drivingand a prevention of any traffic accident. For this, a variety ofassistant devices and a safety device, for example, a safety belt, anairbag, etc. are provided in a vehicle in order to perform thefunctional functions of vehicle component devices.

Furthermore, a device, for example, a black box, is increasinglyinstalled in a vehicle so as to investigate the reasons for a vehicleaccident when a vehicle has an accident in such a manner to store therunning-related images of a vehicle and the data transferred fromvarious sensors. A portable terminal, for example, a smart phone, atablet, etc. is equipped with the function of a black box or anavigation application, etc. and is usually used as one of vehicledevices.

However, the applicability of such running-related images is actuallylow in the vehicle devices. More specifically, even though therunning-related images of a vehicle are obtained through a visionsensor, for example, a camera provided in the vehicle, the electronicapparatus of the vehicle is designed to simply display or transmit suchdata or generate a simple alarming information, for example, a linedeparting state, etc.

In addition, as an electronic apparatus for a vehicle which becomesnewly commercial in recent years, a HUD (Head Up Display) or anaugmented reality interface are proposed, however the applicability ofthe running-related images of a vehicle is a simple display or a simplegeneration of alarming information.

SUMMARY OF THE INVENTION

Accordingly, the present invention is made in an effort to resolve theabove problems. It is an object of the present invention to provide anelectronic apparatus and a control method thereof which provides togenerate a position information of a lane where a vehicle is located,using a driving-related image data of a vehicle and to perform adriving-related guide based on the same.

It is another object of the present invention to provide an electronicapparatus and a control method thereof which provide to effectivelyperform a driving-related guide according to an augmented reality (AR).

To achieve the above objects, there is provided a control method of anelectronic apparatus, comprising: identifying a line region from adriving-related image data of a vehicle; generating a line informationcorresponding to a lane where the vehicle is located from an image dataof the identified line region portion; generating a position informationof a lane where the vehicle is located using at least one of thegenerated line information and the lane information of a road where thevehicle is located; and performing a driving-related guide of thevehicle using the generated lane position information.

In addition, the line information includes a line type information and aline color information which each correspond to the lines formed at bothsides of the lane where the vehicle is located.

In addition, generating the lane position information includes:obtaining a lane information of the road where the vehicle is locatedfrom a map data; determining whether or not the vehicle is located onthe first lane or the last lane of the load using the generated lineinformation; and generating the position information of the lane wherethe vehicle is located by reflecting the lane information of the road,if the vehicle is located on the first lane or the last lane.

In addition, generating the lane position information further includes:wherein if the lane where the vehicle is located changes to the lanepositioning between the first lane and the last lane according to thelane change of the vehicle, updating the generated lane positioninformation with the changed lane position information.

In addition, if the lane where the vehicle is located changes from thelane which is located between the first or last lane to the first laneor the last lane according to the lane change of the vehicle,regenerating the position information of the lane where the vehicle islocated by reflecting the lane information of the road where the vehicleis located.

In addition, the performing the driving-related guide of the vehicleincludes: outputting a lane change guide using the navigation route ofthe vehicle and the lane position information.

In addition, the performing the driving-related guide of the vehicleincludes: outputting a lane guide where the vehicle is located using thelane position information.

In addition, there is further comprising: outputting an appropriate linedeparture guide based on the type of the line, formed at both sides ofthe lane where the vehicle is located which is identified based on theline information.

In addition, the outputting comprises: generating an indicator forperforming the driving-related guide; and outputting the generatedindicator through the AR.

To achieve the above objects, there is provided an electronic apparatus,comprising a line information generation unit which identifies a lineregion from a driving-related image data of a vehicle and generates aline information corresponding to a lane where the vehicle is locatedfrom the image data of the identified line region portion; a laneposition information generation unit which generates a positioninformation of a lane where the vehicle is located using at least onethe generated line information and the lane information of the roadwhere the vehicle is located; and a control unit which performs adriving-related guide of the vehicle using the generated lane positioninformation.

In addition, the line information includes a line type information and aline color information which each correspond to each lines formed atboth sides of the lane where the vehicle is located.

In addition, the lane position information generation unit is configuredto obtain a lane information of the road where the vehicle is locatedfrom a map data, determines using the generated line information,whether or not the vehicle is located on the first lane or the last laneof the road, and if the vehicle is located on the first lane or the lastlane, generates a position information of the lane where the vehicle islocated by reflecting the lane information of the road.

In addition, if the lane where the vehicle is located changes to thelane between the first lane and the last lane according to the lanechange of the vehicle, the lane position information generation unitupdates the generated lane position information with the changed laneposition information.

In addition, if the lane where the vehicle is located changes from thelane which is located between the first or last lanes to the first laneor the last lane according to the lane change of the vehicle, the laneposition information generation unit regenerates a position informationof a lane where the vehicle is located by reflecting the laneinformation of the road where the vehicle is located.

In addition, the control unit controls for the output unit to output alane change guide using the navigation route of the vehicle and the laneposition information.

In addition, the control unit is configured to control the output unitfor output a lane guide where the vehicle is located using the laneposition information.

In addition, the control unit is configured to control the output unitto select and output an appropriate line departure guide based on thetype of the line, formed at both sides of the lane where the vehicle islocated which is identified based on the line information.

In addition, the control unit is configured to generate an indicator forperforming the driving-related guide and control the output unit tooutput the generated indicator through the AR.

To achieve the above object, there is provided a control method of anelectronic apparatus, comprising: receiving a user input which requestsa route guide; generating a route guide line based on a destinationinformation corresponding to the request; correcting the generated routeguide line by reflecting a running trace radius of an actually runningvehicle; performing a variable 3D process by changing, with respect tothe corrected route guide line, the height based on the distance to theuser's vehicle; mapping a texture onto a 3D data generated based on thevariable 3D process, thus generating a route guide indicator; andoutputting the route guide indicator on a screen through the AR.

In addition, the correcting the route guide line includes: addingvertexes to the route guide line of the user's vehicle front section fora straight line; adding vertexes to the curve section of the route guideline for a curve line; and generating the route guide line to which therunning trace radius of the actually running vehicle is reflected byusing the added vertexes.

In addition, the performing the variable 3D process includes: generatinga virtual route guide line at both sides of the corrected route guideline; calculating the height value of the vertexes contained in thecorrected route guide line to increase in proportion to the distance tothe user's vehicle; and performing a 3D process through a polygonizationwith respect to the vertexes of the route guide line the height value ofwhich is calculated, and the vertexes each included in the virtual routeguide line.

In addition, the generating the route guide indicator generates a routeguide indicator by mapping into the 3D data, the texture having adisplacement based on the vehicle speed.

In addition, there are further provided performing a calibration toestimate a camera parameter corresponding to the camera using aphotographed image taken by the camera; and generating a virtual 3Dspace from the photographed image based on the camera parameter, whereinthe outputting comprises: outputting by combining the generated routeguide indicator with the generated virtual 3D space.

To achieve the above objects, there is provided an electronic apparatus,comprising an input unit which receives a user input which requests aroute guide; a route guide line generation unit which generates a routeguide line based on a destination information corresponding to therequest; a route guide indicator generation unit which generates a routeguide indicator for a route guide in the AR using the generated routeguide line; and a display unit which outputs the generated route guideindicator on a screen through the AR, wherein the route guide indicatorgeneration unit includes a route guide line process unit which correctsthe generated guide line by reflecting a running trace radius of theactually running vehicle; a route guide line 3D process unit whichperforms a variable 3D process by changing, with respect to thecorrected route guide line, the height based on the distance to theuser's vehicle; and a texture mapping unit which maps a texture onto thegenerated 3D data based on the variable 3D process, thus generating aroute guide indicator.

In addition, the route guide line process unit adds vertexes to theroute guide line of the user's vehicle front section for a straightline, adds vertexes to the curve section of the route guide line for acurve line, and generates the route guide line to which the runningtrace radius of the actually running vehicle is reflected by using theadded vertexes.

In addition, the route guide line 3D process unit generates a virtualroute guide line at both sides of the corrected route line, calculates aheight value for the height values of the vertexes included in thecorrected route guide line to increase in proportion to the distance tothe user's vehicle, and performs a 3D process, through a polygonizationwith respect to the vertexes of the route guide line the height value ofwhich is calculated, and the vertexes included in the virtual routeguide lines.

In addition, the texture mapping unit generates a route guide indicatorby mapping into the 3D data the texture having a displacement based onthe speed of the vehicle.

In addition, there are further provided a calibration unit whichperforms a calibration to estimate a camera parameter corresponding tothe camera using a photographed image taken by the camera; and a 3Dspace generation unit which generates a virtual 3D space from thephotographed images based on the camera parameter, wherein the displayunit combines the generated route guide indicator with the generatedvirtual 3D space and outputs it.

To achieve the above objects, there is provided a recording medium witha program code which can execute on a computer the above-describedmethods.

According to various embodiments of the present invention, the lineinformation corresponding to the lane where the vehicle is located isgenerated from the image data of the line region portion, and necessaryprocesses may be performed. Therefore, it is possible to perform variousinformation processes including a line interface output, which uses aline information, and an AR interface generation.

In addition, according to various embodiments of the present invention,in case where the center lanes of the road are multiple in number (forexample, in case where the number of lanes is 4 or more than 4), it ispossible to accurately know the lane where the vehicle is located amongthe multiple center lanes.

In addition, according to various embodiments of the present invention,a driver's assistant role can be performed by determining the lane wherethe vehicle is located and guiding the driver.

In addition, according to various embodiments of the present invention,an accurate lane change alarming can be performed using the routeinformation of the navigation and the lane where the vehicle is locatedthus enhancing user's conveniences.

In addition, according to various embodiments of the present invention,an appropriate lane change alarming can be selectively performedaccording to the type of the line at both sides of the lane where thevehicle is running, using the line information, thus enhancing theperformance of the lane change alarming along with enough information.

In addition, according to various embodiments of the present invention,a route guide indicator appropriate to the AR can be constructed in realtime through the 3D process of the route guide line, so the 3D routeguide indicator can be effectively displayed like reality on the 2Dcamera images. Namely, it is possible to express the images wherein theroute guide line positions on the reality-like road, not the display ofthe route guide line in a simple type like in the conventional ARnavigation.

BRIEF DESCRIPTION OF THE DRAWING

FIG. 1 is a block diagram depicting an electronic apparatus according toan embodiment of the present invention.

FIG. 2 is a view for describing a system network connected to anelectronic apparatus according to an embodiment of the presentinvention.

FIG. 3 is a flow chart for describing a line information generationmethod of an electronic apparatus according to an embodiment of thepresent invention.

FIG. 4 is a flow chart for describing in detail a line informationgeneration method of an electronic apparatus according to an embodimentof the present invention.

FIG. 5 is a view illustrating a gray image conversion and a line regiondetection procedure according to an embodiment of the present invention.

FIG. 6 is a view illustrating a line type interested region in a grayimage according to an embodiment of the present invention.

FIG. 7 is a view depicting a line type region-of-interest binarizationand a 1-dimensional mapping in a gray image according to an embodimentof the present invention.

FIG. 8 is a flow chart depicting in detail a lane position informationgeneration method according to an embodiment of the present invention.

FIG. 9 is a view depicting a lane decision table according to anembodiment of the present invention.

FIG. 10 is a view illustrating a lane judgment table according toanother embodiment of the present invention.

FIG. 11 is a flow chart for describing a control method according to anembodiment of the present invention.

FIG. 12 is a bloc diagram illustrating in detail an AR provision unitaccording to an embodiment of the present invention.

FIG. 13 is a view illustrating a comparison before and after a treatmentof a route guide line.

FIG. 14 is a view illustrating a routine guide line 3D process accordingto an embodiment of the present invention.

FIG. 15 is a flow chart for describing an AR route guide methodaccording to an embodiment of the present invention.

FIG. 16 is a view illustrating a route guide screen according to anembodiment of the present invention.

FIG. 17 is a view depicting an implementation form in case where acamera and an electronic apparatus are separate types according to anembodiment of the present invention.

FIG. 18 is a view illustrating an implementation form in case where acamera and an electronic apparatus are integrated types according to anembodiment of the present invention.

FIG. 19 is a view illustrating an implementation form which uses a HUD(Head-UP Display) and an electronic apparatus according to an embodimentof the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

The descriptions hereinafter propose the principles of the invention, soit is obvious that a person having ordinary skill in the art canimplement the principle of the invention even though it is not describedor depicted in the present specification and can invent variousapparatuses included in the concept and scope of the invention. Inaddition, it should be understood that the conditional terms andembodiments listed in the present invention are principally intended tohelp the concepts of the present invention to be fully understood, notlimiting the listed embodiments and states.

In addition, it should be understood that all the detailed descriptionslisting the principle, view point and embodiments as well as specificembodiments of the present invention are intended to include thestructural and functional equivalents of these matters. In addition, itshould be understood that these equivalent matters include all thedevices invented to perform the same functions irrespective of thecurrently known equivalent matters as well as the equivalent matters,namely, structures which will be developed in the future.

Therefore, for example, it should be understood that the block diagramsof the present specification are intended to show the conception viewpoints of the exemplary circuits which embody the principles of thepresent invention. In similar ways, all the flows, state conversiondiagrams, pseudo codes, etc. may be substantially expressed on acomputer readable medium and may represent various processes which canbe executed by a computer or a processor irrespective of whether thecomputer or the processor is clearly illustrated or not.

The functions of various element depicted in the drawings and includingthe processor or the functional blocks indicates in the form of similarconcepts may be executed using an exclusive hardware as well as thehardware which has abilities to execute the related software. When it isprovided by the processor, the above functions may provided by a singleexclusive processor, a single shared processor or a plurality ofindividual processors, and a part of them may be shared.

The correct use of the processor or the term which is suggested as aconcept similar therewith should not be interpreted in such a way toexclusively cite the hardware which has an ability to execute software,and it should be interpreted that it is indented to implicitly includeROM, RAM and a nonvolatile memory. Well known other hardware may beincluded.

In the claims of the present specification, the components expressed asa method for executing the functions recited in the detaileddescriptions are intended to include all the methods for executing thefunctions which include all types of software including a combination ofcircuit elements performing, for example, the above functions or afirmware/micro code, etc. and may be combined with an appropriatecircuit to execute the software. It should be understood that thepresent invention defined by such claims is combined with the functionsprovided by variously listed means and with the ways that the claimsrequire, so any means for providing the above functions should beunderstood to be equivalent to what can be recognized from the presentspecification.

The above-described objects, features and advantages could become clearwith the aid of the following descriptions in relation with theaccompanying drawings, and a person having ordinary skill in the art towhich the present invention pertains can easily implement the technicalconcepts of the present invention. In addition, while the presentinvention is being described, if it is judged that the descriptions withrespect to the known technology in relation with the present inventionmay make unclear the subject matters of the present invention, suchdetailed descriptions would be omitted.

Various embodiments of the present invention will be described withreference to the accompanying drawings.

FIG. 1 is a block diagram illustrating an electronic apparatus accordingto an embodiment of the present invention. Referring to FIG. 1, theelectronic apparatus 100 includes part or all of a storing unit 110, aninput unit 120, a line information generation unit 140, a lane positioninformation generation unit 150, an AR provision unit 160, a controlunit 170, a communication unit 180, and a sensing unit 190.

Here, the electronic apparatus 100 may be implemented using a smartphone, a tablet computer, a palm computer, a PDA (Personal DigitalAssistant), a PMP (Portable Multimedia Player), a smart glass, a projectglass, a navigation, a black box, etc. each of which may provide to avehicle driver a driving-related information.

Here, the driving state of the vehicle may include, not limited to, avariety of states where the vehicle is being driven by a driver, forexample, a stopping state of the vehicle, a running state of thevehicle, a parking state of the vehicle, etc.

The driving-related guide may include, not limited to, a variety ofguides to assist the driving of a vehicle driver, for example, a roadguide, a line departure guide, a forward vehicle starting guide, asignal change guide, a forward vehicle collision guide, a lane changeguide, a lane guide, etc.

Here, the route guide may include, but limited to, an augmented realityroad guide performing road guide by combining various information, forexample, a user's position, direction, etc. with the images of theforward scenes of the currently driving vehicle, and a 2D(2-Dimensional) or 3D (3-Dimensional) road guide performing a road guideby combing the map data of the 2D or 3D with various information, forexample, a user's position, direction, etc. Here, the route guide may beinterpreted as an occasion wherein a driver drives on the vehicle aswell as a concept including a road guide where a user moves walking orrunning.

In addition, the line departure guide may be a guide to guide whetherthe running vehicle departs from the line or not.

In addition, the forward vehicle start guide provides to guide whetheror not a vehicle positioned in front of a vehicle which remains stopped,has started.

In addition, the signal lamp change guide provides to guide whether ornot the signal lamp which positions in front of the vehicle whichremains stopped, has changed. As an example, in a state where the redlamp representing the stop signal is on, when the lamp changes to thegreen lamp which means a start signal, such an operation may be guided.

In addition, the forward vehicle collision prevention guide provides toprevent any collision with the forward vehicle when the distance to thevehicle in front of the vehicle which remains stopped or runs is withina predetermined distance.

In addition, the lane change guide may a guide for guiding a change fromthe line where the vehicle is positioned to another line for the sake ofa route guide to the destination.

In addition, the lane guide may be a guide for guiding the line wherethe vehicle is currently positioned.

The driving-related images which help various driving guide functionsmay be photographed in real time by the camera hung in the forwarddirection. Here the camera may be a camera which may be integrallyformed with the electronic apparatus 100 hung in the vehicle so as tophotograph the forward scenes of the vehicle. In this case, the cameramay be integral with the smart phone, the navigation or the black box,and the electronic apparatus 100 may receive the images photographed bythe integral camera.

As another example, the camera may be hung different from the electronicapparatus 100 and may photograph the forward scenes of the vehicle. Inthis case, the camera may be a separate black box which hung for theforward direction of the vehicle, and the electronic apparatus 100 mayreceive the photographed images according to a wired/wirelesscommunication with the separately hung black box or may receive thephotographed images when a storing medium for storing the photographedimages of the black box is inserted in the electronic apparatus 100.

Hereinafter, the electronic apparatus 100 according to an embodiment ofthe present invention will be described in more details according to theabove contents.

The storing unit 110 has a function for storing various data andapplications which are necessary for the operations of the electronicapparatus 100. In particular, the storing unit 110 may store the datanecessary for the operations of the electronic apparatus 100, forexample, OS, a route search application, a map data, etc. In addition,the storing unit 110 may store the data generated by the operations ofthe electronic apparatus 100, for example, a searched route data, areceived image, etc. In addition, the storing unit 110 may store a lanejudgment table, etc.

Here, the storing unit 110 may be implemented using a built-in typestoring element, for example, RAM (Random Access Memory), Flash Memory,ROM (Read Only Memory), EPROM (Erasable Programmable ROM) EEPROM(Electronically Erasable and Programmable ROM), a register, a hard disk,a removable disk, a memory card, USIM (Universal Subscriber IdentityModule), etc. or a detachable type storing element, for example, a USBmemory, etc.

The input unit 120 has a function for converting physical input from theoutside of the electronic apparatus 100 into a specific electricalsignal. Here, the input unit 120 may be all or part of a user input unit121 and a microphone unit 123.

The user input unit 121 may receive a user's input, for example, atouch, a push operation, etc. Here, the user input unit 121 may beimplemented using at least one among a various button types, touchsensor which receive touch inputs, and a near sensor for receiving anapproaching motion.

The microphone unit 123 may receive a user's voice and sound from theinside or outside of the vehicle.

The output unit 130 is a device for outputting the data of theelectronic apparatus 100. Here, the output unit 130 may be all or partof a display unit 131 and an audio output unit 133.

The display unit 131 is a device for outputting data that the electronicapparatus 100 may visually recognize. The display unit 131 may beimplemented with a display unit provided at a front side of the housingof the electronic apparatus 100. Here, the display unit 131 is formedintegral with the electronic apparatus 100 and may output a visualrecognition data and is installed separate from the electronic apparatus100 like the HUD and may output a visual recognition data.

The audio output unit 133 is a device for outputting the data that theelectronic apparatus 100 may audibly recognize. The audio output unit133 may be formed of a speaker which may output in the forms of soundthe data which should be informed to the user of the electronicapparatus 100.

The communication unit 180 may provide to communicate with anotherdevice. The communication unit 180 may include, not limited to all orpart of a position data unit 181, a wireless internet unit 183, abroadcast transceiver unit 185, a mobile communication unit 186, a localarea communication unit 187 and a wired communication unit 189.

The position data unit 181 is a device for obtaining the position datathrough the GNSS (Global Navigation Satellite system). The GNSS means anavigation system which may calculate the position of the receiverterminal using radio signals from the satellite. As an example of theGNSS, there may, according to its operation body, be GPS (GlobalPositioning System), Galileo, GLONASS (Global Orbiting NavigationalSatellite System), COMPASS, IRNSS (Indian Regional NavigationalSatellite System), QZSS (Quasi-Zenith Satellite System), etc. Theposition data unit 181 of the electronic apparatus 100 according to anembodiment of the present invention may obtain a position data byreceiving a GNSS signal which is served in a region where the electronicapparatus 100 is being used.

The wireless internet unit 183 is a device for connecting to thewireless internet, thus obtaining or transmitting the data. The wirelessinternet unit through the wireless internet unit 183 for connecting tothe wireless internet may include, but not limited to, WLAN (WirelessLAN), Wibro (Wireless broadband), Wimax (World interoperability formicrowave access), HSDPA (High Speed Downlink Packet Access), etc.

The broadcast transceiver unit 185 is a device for transmitting andreceiving broadcast signals through various broadcast systems. Thebroadcast system for transmitting and receiving through the broadcasttransceiver 185 may include, but not limited to, DMBT (DigitalMultimedia Broadcasting Terrestrial), DMBS (Digital MultimediaBroadcasting Satellite), MediaFLO (Media Forward Link Only), DVBH(Digital Video Broadcast Handheld), ISDBT (Integrated Services DigitalBroadcast Terrestrial), etc. The broadcast signal which may betransmitted or received through the broadcast transceiver unit 185 mayinclude, but not limited to, a traffic data, a life data, etc.

The mobile communication unit 186 may communicate by connecting to amobile communication network in compliance with various mobilecommunication criteria, for example, 3G (3rd Generation), 3GPP (3rdGeneration Partnership Project), LTE (Long Term Evolution), etc.

The local area communication unit 187 is a device for the sake of alocal area communication. The local area communication unit 187 mayprovide to communicate through Bluetooth, RFID (Radio FrequencyIdentification), infrared ray communication (IrDA, Infrared DataAssociation), UWB (Ultra WidBand), ZigBee, NFC (Near FieldCommunication), Wi-Fi, etc.

The wired communication unit 189 is an interface device for connectingthe electronic apparatus 100 to another device through a wiredconnection. The wired communication unit 189 may be a USB module whichmay communicate through the USB port.

The communication unit 180 may communicate with another device using atleast one of the position data unit 181, a wireless internet unit 183, abroadcast transceiver unit 185, a mobile communication unit 186, a localarea communication unit 187, and a wired communication unit 189.

As an example, in case where the electronic apparatus 100 does notinclude a camera function, using at least one of the local areacommunication unit 187 and the wired communication unit 189, it ispossible to receive the images taken by the vehicle camera, for example,a black box, etc.

As another example, in case where a communication is made to multipledevices, any one of them communicate with the local area communicationunit 187, and the other one of them may communicate through the wiredcommunication unit 189.

The sensing unit 190 is a device for detecting the current state of theelectronic apparatus 100 and may include, but not limited to, all orpart of a motion sensing unit 191 and a light sensing unit 193.

The motion sensing unit 191 may detect the motion in the 3D space of theelectronic apparatus 100. The motion sensing unit 191 may be a 3-axisgeomagnetic sensor and a 3-axis acceleration sensor. The motion dataobtained by the motion sensing unit 191 is combined with the positiondata obtained by the position data unit 181, thus calculating a moreaccurate trace than the vehicle with the electronic apparatus 100.

The light sensing unit 193 is a device for measuring surroundingilluminance of the electronic apparatus 100 and allows to change thebrightness of the display unit 131 to the surrounding brightness usingthe illuminance data obtained through the light sensing unit 193.

The power unit 195 is a device for supplying power which is necessaryfor the operation of the electronic apparatus 100 and the operation ofanother device connected to the electronic apparatus 100. The power unit195 may be a device for receiving power from external power source, forexample, a battery in the electronic apparatus 100 or a vehicle. Inaddition, the power unit 195 may, depending on the type for receivingpower, be implemented in the form of a wired communication module 119 ora device for receiving power through a wireless connection.

Meanwhile, the control unit 170 control the whole operations of theelectronic apparatus 100. In more detail, the control unit 170 maycontrol all or part of the storing unit 110, the input unit 120, theoutput unit 130, the line information generation unit 140, the laneposition generation unit 150, the AR provision unit 160, thecommunication unit 180 and the sensing unit 190.

In particular, the control unit 170 may control the line informationgeneration unit 140 and the lane position information generation unit150 to identify a line region from the driving-related image data, togenerate a line information corresponding to the lane where the vehicleis located from the image data of the identified line region portion,and to generate a lane position information where the vehicle is locatedusing at least one of the generated line information and the laneinformation of the road where the vehicle is located.

Here, the line may mean both lines which define a lane where the vehicleis located. The lane may be formed by the lines into the forms of afirst lane, a second lane, . . . , N-th lane and may mean a road wherethe vehicle can run.

The line information generation unit 140 may identify a line region fromthe image data taken in a driving state, for example, a driving, stop,etc. of the vehicle, and generate a line information corresponding eachline formed at both sides of the lane where the vehicle is located fromthe image data of the line region portion. The line information mayinclude a line type information and a line color information eachcorresponding to each line formed at both sides of the lane where thevehicle is located.

Here, in order to generate a line information corresponding to the lanewhere the vehicle is located, the line information generation unit 140may perform a binarization with respect to the image data of the lineregion portion and obtains a line type information from the binarizedpartial image data. In detail, the line information generation unit 140analyzes the binarized partial image data using at least one of the timecontinuity information of the line and the line speed information, thusidentifying whether or not the kinds of both the lines of the lane wherethe vehicle is located is a solid line or a dotted line.

In addition, the line information generation unit 140 may extract froman image data a color information corresponding to each line the kind ofwhich was identified, thus generating a line information.

The lane position information generation unit 150 may generate aposition information of a lane where the vehicle is located according tocontrolling of the control unit 170, using at least one of the lineinformation generated by the line information generation unit 140 andthe lane information of the road where the vehicle is located.

More specifically, the lane position information generation unit 150 mayobtain a lane information of the road where the vehicle is located fromthe map data, and judges whether or not the vehicle is located on thefirst lane or the last lane of the road using the generated lineinformation, and if the vehicle is located on the first lane or the lastlane, generates a lane position information where the vehicle is locatedby reflecting the lane information of the road. Thereafter, if the lanewhere the vehicle is located changes to the lane next to the first laneor the last lane, due to the change of the lane of the vehicle, thegenerated lane position information may be updated using the changedlane position information. Thereafter, if the lane where the vehicle islocated changes due to the change of the lane of the vehicle from thelane between them to the first lane or the last lane, a positioninformation of a lane where the vehicle is located can be regenerated byreflecting the lane information of the road where the vehicle islocated.

Here, the lane information of the road where the vehicle is located mayinclude a lane count information of the road where the vehicle islocated and a road type information (for example, an expressway, a cityexpressway, a regional road, a common road) and may be obtained from themap data stored in the storing unit 110 of the electronic apparatus 100,a database (DB) of an external map separate from the electronicapparatus 100, or another electronic apparatus 100. For example, if theelectronic apparatus 100 is implemented in a form of a black box, theblack box may provide to obtain a lane information of a road from anexternal navigation device which may be connected communicated.

Meanwhile, the lane information that each country uses are different.The lane position information generation unit 150 can generate positioninformation using the lane judgment table in compliance with the trafficregulation, etc. by country. So, the lane position information may begenerated according to the lane judgment table mapped on the countryinformation set by the lane position information generation unit 150.

Meanwhile, the control unit 170 may perform the driving-related guide ofthe vehicle using the line information generation unit 140 and the laneposition information generation unit 150.

As an example, the control unit 170 may select an appropriate linedeparture guide according to the type of the line, formed at both sidesof the lane where the vehicle is located , which is identified on basisof the line information and may output to the user. In detail, thecontrol unit 170, if the electronic apparatus 100 provides a linedeparture guide function, may provide different guides according to theline type and color of the departed line. For example, the control unit170 may select and output different guide images or guide voicesaccording to whether or not the vehicle has intruded the central line,has intruded the white solid line, has intruded the white dotted line orhas intruded the blue line.

As another example, the control unit 170 may output the generated lineguide through the output unit 140 using the lane position informationgenerated by the lane position information generation unit 150. Indetail, the control unit 170 may output which number the vehicle islocated on, for example, the recognition of the first lane, the secondlane, . . . , the N-th lane, in a form of image or voice.

As further another example, the control unit 170 may output thegenerated lane change guide through the output unit 140 using the laneposition information generated by the lane position informationgeneration unit 150. In detail, the control unit 170, if the electronicapparatus 100 provides a navigation function of the vehicle, may outputin a form of image or guide voice the lane change guide based on theroute to the destination and the judged lane position. Namely, thecontrol unit 170 may judge whether or not the current lane is allowedfor the left turn or the right turn if the distance to the left turn orright turn guide spot is less than a predetermined distance and outputsa lane change guide to the user.

Meanwhile, the control unit 170 may control the AR provision unit 160for the electronic apparatus 100 to perform the driving-related guidebased on the AR. Here, the AR may be a way to visually show in duplicatethe added information (for example, a graphic component which representsthe point of interest (POI), a graphic component which represents theroute to the destination, etc.) on the screen which contains the realworld that the user is actually seeing. In this case, the control unit170 may generate an indicator for performing the driving-related guidein cooperation with the AR provision unit 160 and may output thegenerated indicator through the output unit 130. the augmented realitymay be provided using the HUD which uses the wind shield of the vehicleor an image overlay which uses a separate image output device. Theaugmented reality provision unit 160 may generate a real image and aninterface image, etc. which overlaps on the glass. Based on the abovefeatures, it is possible to implement an augmented reality navigation ora vehicle infortainment system.

In particular, according to an embodiment of the present invention, aroute guide indicator may be constructed in real time, which is properto the AR through the 3D process of the route guide line. It is possibleto effectively display a 3D route guide indicator on the camera image ofthe 2D, thus providing a reality-like indicator, which will be describedlater in more detail.

FIG. 2 is a view for describing a system network which is connected toanother electronic apparatus according to an embodiment of the presentinvention. Referring to FIG. 2, the electronic apparatus 100 of anembodiment of the present invention may be implemented using variousdevices provided in the vehicle, for example, a navigation, a black box,a smart phone or other vehicle AR interface provision device and may beconnected with various communication networks and other electronicdevices 61 to 64.

In addition, the electronic apparatus 100 may calculate the currentposition and current timing in cooperation with a GPS module inaccordance with a radio signal from the satellite. Each satellite 20 maytransmit or receive L-band frequencies the frequency bands of which aredifferent. The electronic apparatus 100 may calculate the currentposition according to the time which has lapsed until the L-bandfrequency from each satellite 20 reaches the electronic apparatus 100.

Meanwhile, the electronic apparatus 100 may connect to the network 30 ina wireless way through the communication unit 180 and through thecontrol station 40 (ACR) and the base station 50 (RAS). When theelectronic apparatus 100 gets connected to the network 30, it mayindirectly connected to the electronic devices 61 and 62, thusexchanging the data.

Meanwhile, the electronic apparatus 100 may indirectly connect to thenetwork 30 through another device 63 which has a communication function.For example, in case where the electronic apparatus 100 is not equippedwith a module which may connect to the network 30, it is possible tocommunicate with another device 63 which has a communication functionthrough the local area communication module.

FIG. 3 is a flow chart for describing a line information generationmethod of an electronic apparatus according to an embodiment of thepresent invention. Referring to FIG. 3, the electronic apparatus 100 mayidentify a line region from the driving-related image data of thevehicle (S101). In more detail, the line information generation unit 140may convert a driving-related image into a gray image and determine as aline region which is formed at both sides of a vehicle by performing aline detection algorithm. Here, the driving-related image of the vehiclemay include the images related to a stop, a running, etc. of thevehicle. In addition, the driving-related image of the vehicle may be animage taken by a camera module of the electronic apparatus 100 or animage that the electronic apparatus 100 has received from anotherdevice. In addition, the driving-related image of the vehicle may be RGB(Red, Green, Blue) colors.

In addition, the electronic apparatus 100 may generate a lineinformation corresponding to the lane where the vehicle is located fromthe image data of the identified line region portion (S102). In detail,the line information generation unit 140 may generate a line informationby analyzing the pattern information and color information of the linewith respect to the detected line region. The line information mayinclude at least one of the line type information and the line colorinformation which correspond to each line positioned at both sides ofthe lane where the vehicle is located.

Hereinafter, the line information generation method will be described indetail with reference to FIGS. 4 to 7.

FIG. 4 is a flow chart for describing in detail the line informationgeneration method of the electronic apparatus according to an embodimentof the present invention. Referring to FIG. 4, the electronic apparatus100 may convert a color image data into a gray image (S201), and mayperform a line region detection from the converted gray image (S202).

In more detail, the line information generation unit 140 may extract aregion for detecting lines from the photographed driving-related images.Also, the line information generation unit 140 may previously perform alight source compensation with respect to the original image in order tominimize any shadow effects since it is hard to detect lines if part ofthe road is affected by the shadow.

And, the line information generation unit 140 may detect as a lineregion the region where the line may exist based on the previouslydetermined position of the camera or the installation angle of thecamera. As an example, the line information generation unit 140 maydetermine the line region about a position where the line may begin. Inaddition, the line information generation unit 140 may estimate theposition where the line region begins and the length of the line regionas the width (the maximum width between the left line region and theright line region) in the driving-related image and the viewing angle ofthe camera.

Also, the line information generation unit 140 may convert the grayimage corresponding to the line detection region into an edge image andmay detect the line region based on the straight position extracted fromthe converted edge image. More specifically, the driving-related imagemay be converted into an edge image through various known algorithms.The edge image may include an edge which shows multiple straight lines.At this time, the line information generation unit 140 may recognize asa line the detected straight line. Also, the line information generationunit 140 may determine the line region based on the position of thestraight line which has the width of a constant line width with respectto the running direction of the vehicle among multiple straight linecandidates.

FIG. 5 shows the above gray image conversion and the line regiondetection procedure. Referring to FIG. 5, the inputted driving-relatedcolor image 203 may be converted into a gray image 200, and line regions201 and 202 of a straight line may be detected through a line detectionalgorithm of an edge detection, etc. The line region may be categorizedinto a left line region 201 and a right line region 202 about theposition of the vehicle.

Turning back to FIG. 4, it will be described.

Thereafter, if the line region is detected, the line informationgeneration unit 140 may set a line type region-of-interest region basedon the line region (S203). In detail, if the line region is detected,the line information generation unit 140 may set a line typeregion-of-interest region (ROI) according to the detected line region.The line type region-of-interest region may mean the portion of thedriving-related image which includes the line and its surrounding regionso as to judge the types and colors of the lines.

More specifically describing, FIG. 6 shows a line typeregion-of-interest in the gray image.

As illustrated in FIG. 6, the line type region-of-interests 210 and 220may partially include the previously detected line region and itsperipheral region. In addition, the line type region-of-interest may becategorized about the proceeding direction of the vehicle into a leftline type region-of-interest 210, and a right line typeregion-of-interest 220.

As an example, if the detected line region is formed like a linear lineand is expressed like y=a*x+b, the line type region-of-interest may be aregion including both y=a*x+b+m and y=a*x+b−m. Different from aconventional simple line detection method, as a method for generatingdetailed and various running line information, the line informationgeneration unit 140 expands the line region of the detected straightline and may set its peripheral region as a region-of-interest.

Turning to FIG. 4, it will be described.

Thereafter, the line information generation unit 140 performs abinarization with respect to the line type region-of-interest region(S204) and maps the binarized partial gray image to a 1D region (S205)and can identify the types of lines using at least one of the timecontinuity and speed (S206).

The line information generation unit 140 may extract a partial grayimage of the line type region-of-interest region from the converted grayimage and performs a binarization with respect to the partial grayimage. So, the line information generation unit 140 can clearlydistinguish only the portion judged to be a line, from the partial grayimage.

And, the line information generation unit 140 can map, to a 1D region,each line(left line and right line) which can be recognized in thebinarized partial gray image. In addition, the types of lines can beidentified by analyzing the pattern of each line mapped to the 1Dregion.

In more detail, FIG. 7 depicts a line type region-of-interestbinarization and a 1D mapping in the gray image.

As depicted in FIG. 7, if a binarization with respect to the line typeregion-of-interest is performed, a binarized image 300 may be obtained.In the binarized image 300, the portion appearing in white may beidentified as a line, the other portions may be identified as black.

And, each line identified in the binarized image 300 may be mapped to a1D region. The line information generation unit 140 may easily identifythe types of lines using the image 310 mapped to the 1D region.

For example, the line information generation unit 140 may judge thedotted line or solid line based on the start point and length feature ofeach line mapped to the 1D. Also, the line information generation unit140 may judge the dotted line or solid line using continuity withrespect to time of each line mapped to the 1D and using the speed. And,the line information generation unit 140 firstly determines the dottedline or solid line based on the start point and length feature and thensecondarily determines the dotted line or solid line using the timecontinuity and speed.

More specifically, the line information generation unit 140 maydetermine the dotted line or solid line by comparing and judging theline length based on the position of the start point of each line. Inthis case, the line information generation unit 140 may judge the dottedline or solid line using one image frame.

Also, the line information generation unit 140 may clearly judge if aline is a dotted line or a solid line depending on whether each line iscontinuously formed or not as time passes(S206). As an example, the lineinformation generation unit 140 may previously sets a continuity levelaccording to the moving speed of the line in the image and may judge asa dotted line if the continuity of each line is lower than the value ofa previously set continuity level.

Therefore, according to the embodiment of the present invention, thedotted line or the solid line is previously identified using one frame,and such identification can be proved using a continuous frame, thusfinally judging the type of line.

Referring back to FIG. 4, the electronic apparatus 100 may detect acolor of a line portion the type of which has been identified, from thecolorful original image data (S207).

The line information generation unit 140 may detect the color at aportion corresponding to the line the type of which has been identified,by analyzing a color image and sorts the same. As an example, the lineinformation generation unit 140 can sort the detected color into white,yellow or blue.

Thereafter, the electronic apparatus 100 may generate a line informationcorresponding to the lane where the vehicle is located according to theidentified line type and classified color (S208).

FIG. 8 is a flow chart depicting in detail the method for generating alane position information according to an embodiment of the presentinvention. Referring to FIG. 8, the electronic apparatus 100 can obtaina lane information of the road where the vehicle is located, from themap data (S301). Here, the lane information of the road may be a laneinformation of the road where the running vehicle is currently locatedor may include an information on the number of the lanes of the roadwhere the vehicle is located. In addition, the lane information of theroad may be obtained from the map data stored in the storing unit 110 inthe electronic apparatus 100 or may be obtained from an external mapdatabase (DB) being separate from the electronic apparatus 100 or may beobtained from another electronic apparatus 100.

And, the electronic apparatus 100 may judge, using the generated lineinformation, whether the vehicle is located on the first lane of theroad or on the last lane (S302). More specifically, the lane positioninformation generation unit 150, as depicted in FIG. 9, judges whetherthe vehicle is located on the first lane of the road or on the last laneof the road by reflecting the line information corresponding to the lanein the lane judgment table.

Namely, the lane judgment table may contain the first lane and the lastlane which are determined according to the nation, the line types andcolors of the left line, the types and colors of the right line. Here,the lane judgment table as depicted in FIG. 10 may be provided asexamples, and may be set as different values depending on nations orsituations for each setting.

Meanwhile, the electronic apparatus 100 may generate the positioninformation of the lane where the vehicle is located by reflecting thelane information of the road, if the vehicle is located on the firstlane or the last lane (S303). As an example, if it is judged that thevehicle is in the last lane, the lane position information generationunit 150 may generate the lane position information as a N-lane. Inaddition, if the number of the lanes corresponding to the road laneinformation is 5, the N-lane may be generated into 5th lane based on theabove principle.

And, the electronic apparatus 100 may update the lane positioninformation by using the changed lane position information, if the lanewhere the vehicle is located changes to the lane between the first laneand the last lane according to the lane change of the vehicle (S304). Inthis case, the lane position information generation unit 150 may judgethe departing of the line by using the line information and judges thechanges of the lane based on the above judgment. As an example, the laneposition information generation unit 150 may update the lane positioninformation from the 5th lane to the 4th lane if it is judged that thevehicle has changed from the 5th lane to the left side lane, which meansthe lane change by one lane.

And, the electronic apparatus 100 may obtain again the lane informationof the road where the vehicle is located, if the lane where the vehicleis located changes from the lane which is located between the first orlast lanes to the first lane or last lane according to the lane changeof the vehicle (S305). In addition, the lane position informationgeneration unit 150 may re-generate the lane position informationcorresponding to the lane where the vehicle is located, by reflectingthe re-obtained lane information (S306). As an example, if it is judgedthat the vehicle changes from the 4th lane to the right side lane, whichmeans the change by one lane, the vehicle has changed to the 5th lanewhich is the previously set last lane, thus obtaining the laneinformation of the road where the vehicle is currently located. Inaddition, if an obtained lane information is the 4th lane, the positioninformation of the lane where the vehicle is located may be re-generatedas the 4th lane.

Meanwhile, the lane position information generation method according toan embodiment of the present invention is not limited to FIG. 9.Therefore, the above-described sequences may change according to anotherembodiment. As an example, the step wherein the lane information of theroad where the vehicle is located may be obtained in the step S304. Inthis case, the lane position information generation unit 150 maygenerate the position information of the lane where the vehicle islocated if the vehicle is located on the first or last lane. As anexample, if it is judged that the vehicle is located on the last lane,the lane position information generation unit 150 can generate the laneposition information as the N- lane.

And, the electronic apparatus 100 may update the lane positioninformation using the generated lane position information and theobtained lane information of the road, if the lane where the vehicle islocated changes to the lane which is located between the first or lastlanes according to the lane change of the vehicle (S304). As an example,the lane position information generation unit 150 may update the laneposition information to the 4th lane in such a manner to reflect N=5which is the information corresponding to the number of lanes to the N−1lane if it is judged that the vehicle has changed from the N-lanecorresponding to the last lane to the left lane, which means the changeby one lane.

And, according to another embodiment of the present invention, the laneposition information generation unit 150 may judge if the vehicle islocated on the first lane, the center lane or the last lane of the roadby applying the line information corresponding to the lane where thevehicle is located to the lane judgment table as depicted in FIG. 10.However, in this case, it is hard to accurately know the lane where thevehicle is located among the multiple center lanes if the center lane ofthe road is provided multiples in number (for example, the number oflanes is 4 or more than 4). The embodiment of the present invention maypreferably use the method as depicted in FIG. 9.

FIG. 11 is a flow chart for describing a control method according to anembodiment of the present invention. Referring to FIG. 11, theelectronic apparatus 100 may identify a line region from thedriving-related image data of the vehicle(S401).

In addition, the line information corresponding to the lane where thevehicle is located may be generated from the image data of theidentified line region portion (S402).

In addition, the lane position information where the vehicle is locatedmay be generated using at least one of the generated line informationand the lane information of the road where the vehicle is located(S403).

In addition, the driving-related guide of the vehicle may be performedusing the obtained lane information (S404).

Here, the step S404 for performing the driving-related guide of thevehicle may include outputting a lane change guide using the navigationroute of the vehicle and the lane position information.

In addition, the step S404 for performing the driving-related guide ofthe vehicle may include outputting a lane guide where the vehicle islocated using the lane position information.

Meanwhile, the control method of an embodiment of the present inventionmay include selecting and outputting an appropriate line departure guideaccording to the type of the line at both sides of the lane of thevehicle recognized based on the line information.

Here, the outputting may be performed by generating an indicator forperforming the driving-related guide and outputting the generatedindicator through the AR.

Meanwhile, the conventional AR navigation uses various techniques interms of the route guide, but there is a limit to express the routeguide due to a problem in mixing the route guide line and the actualroad environment.

However, according to an embodiment of the present invention, anappropriate route guide indicator matching with the AR can beconstructed in real time based on the 3D processing of the route guideline, thus effectively expressing 3D route guide indicator, which lookslike reality, on the 2D camera images. The AR provision unit 160according to an embodiment of the present invention for achieving theabove objects will be described in detail below.

FIG. 12 is a block diagram illustrating in detail the AR provision unit160 according to an embodiment of the present invention. Referring toFIG. 12, the AR provision unit 160 may include a part of all of acalibration unit 161, a 3D space generation unit 162, an indicatorgeneration unit 163, and a mapping unit 164.

The calibration unit 161 may perform calibrations for estimating acamera parameter corresponding to the camera from the photographedimages taken by the camera. Here, the camera parameter may be aparameter which forms a camera matrix which is an information showing arelationship between a real space and a photo.

The 3D space generation unit 162 may generate a virtual 3D space basedon the photographed images taken by the camera. In detail, the 3D spacegeneration unit 162 may obtain a depth information from the image takenby the camera based on the camera parameter estimated by the calibrationunit 161 and may generate a virtual 3D space based on the obtained depthinformation and the photographed image.

The indicator generation unit 163 may generate an indicator for guidesin the AR reality, for example, a route guide indicator, a lane changeguide indicator, a line departure guide indicator, etc.

In particular, when a user input requesting a route guide to thedestination is received through the input unit 120, the indicatorgeneration unit 163 may generate a route guide indicator for a routeguide on the AR. Here, the route guide indicator generation unit mayinclude a route guide line process unit 163-1, a route guide line 3Dprocess unit 163-2 and a dynamic texture mapping unit 163-3.

When a route guide line to the destination is generated at a user'sroute guide request, the route guide line process unit 163-1 may performthe processing of a route guide line by reflecting a running traceradius of an actual running vehicle.

In detail, the electronic apparatus 100 may generate a route guide lineto the destination by using a map data obtained from the storing unit110, or a map data obtained from an external map database (DB) which isseparate from the electronic apparatus 100. Here, the generated routeguide line may be formed of a node and a link, as depicted on the leftside of FIG. 13. Referring to the left side of FIG. 13, the generatedroute guide line 1301 may be a straight line which is not similar withthe vehicle running trace of an actual running vehicle in the curvesection 1302. if the generated route guide line 1301 is mixed with thecamera images to provide AR the result of AR screen is different fromthe running trace of the actual running vehicle.

Therefore, the route guide line process unit 163-1 according to anembodiment of the present invention may perform the processing of aroute guide line by reflecting the running trace radius of the actualrunning vehicle. In detail, the route guide line process unit 163-1 mayremove a vertex with respect to the region not shown on the currentscreen among the generated route guide lines, and any unnecessary vertexlike a duplicate spot. In addition, the route guide line process unit163-1 may add vertexes to the user's vehicle front section of the routeguide line so as to form the user's vehicle front section of the routeguide line as a straight line. In addition, the route guide line processunit 163-1 may add vertexes to the curve section of the route guide lineso as to form the curve section of the route guide line as a curve line.In addition, the route guide line process unit 163-1 may generate theroute guide line to which the running trace of an actually runningvehicle has been reflected, by using the added vertexes.

According to an embodiment of the present invention, the route guideline may be generated, as depicted on the right side of FIG. 13.Referring to the right side of FIG. 13, the processed route guide line1303 may be a smooth type which is similar with the the vehicle runningtrace of an actual running vehicle in the curve section 1304.

The route guide line 3D process unit 163-2 may perform a variable 3Dprocess with respect to the route guide line generated by the routeguide line process unit 163-1 while changing the height based on itsdistance. The above operation will be described in detail with referenceto FIG. 14.

Referring to FIG. 14, the route guide line 3D process unit 163-2 maygenerate virtual route guide lines 1402 and 1403 at both sides of theroute guide line 1401 that is generated by the route guide line processunit 163-1. In detail, the route guide line 3D process unit 163-2 maycalculate a unit vector of the vertex of the route guide line 1401generated by the route guide line process unit 163-1 and may calculate anormal vector which is vertical to the unit vector based on the internalcomputation of the calculated unit vector, thus generating virtual routeguide lines 1402 and 1403.

In addition, the route guide line 3D process unit 163-2 may calculatethe height values of the vertexes included in the route guide line 1401.In this case, the route guide line 3D process unit 163-2 may calculatethe height values of the vertexes contained in the route guide line 1401to increase in proportion to the distance to the user's vehicle.

In addition, the route guide line 3D process unit 163-2 may generate,through polygonization, surfaces with respect to the vertexes of theroute guide line 1401 the height value of which has been calculated, andthe vertexes included in the virtual route guide lines 1402 and 1403,thus performing a 3D process.

Therefore, the route guide indicator at the spot which is far from theuser's vehicle among the route guide indicator displayed on the screenmay be displayed on the screen so that the driver can recognize.

The dynamic texture mapping unit 163-3 may map the textures having adisplacement based on the speed of the vehicle onto the 3D datagenerated by the route guide line 3D process unit 163-2.

Here, the textures may be textures having a displacement based on thespeed of the vehicle. Namely, if the vehicles moves along the route, thedynamic texture mapping unit 163-3 may generate a texture having adisplacement based on the speed of the vehicle by changing the mappingposition of the texture in the 3D data. In this case, the route guideindicator displayed on the screen may allow to maximize the effects, forexample, looking like sticking on the surface of the road.

Through this operation, the indicator generation unit 163 may generate aroute guide indicator for the route guide on the AR.

Meanwhile, in the display technique which, assuming that there is a turnpoint, in general is used after a turn point assuming, for the sake of amore effective screen display, the route guide line 3D process unit163-2 may make vertical the 3D images generated based on the aboveoperations.

Namely, in case of the left turn, the route guide line 3D process unit163-2 may determine the 3D data by erecting in the height direction thevirtual route guide line 1403 corresponding to the right side of theroute guide line 1401. On the contrary, in case of the right turn, the3D data may be determined by erecting in the height direction thevirtual route guide line 1402 corresponding to the left side of theroute guide line 1401.

Meanwhile, the mapping unit 164 may combine the indicator generated bythe indicator generation unit 163 with the virtual 3D space generated bythe 3D space generation unit 162.

FIG. 15 is a flow chart illustrating an AR (Augmented Reality) routeguide method according to an embodiment of the present invention.Referring to FIG. 15, the electronic apparatus 100 may receive a userinput which requests a route guide (S501).

In addition, the electronic apparatus 100 may generate a route guideline based on the destination information for a route guide (S502).

In addition, the electronic apparatus 100 may correct the generatedroute guide line by reflecting the running trace radius of the actualrunning vehicle (S503). In detail, the step S503 may include a stepwherein the vertex with respect to the region not shown on the currentscreen among the generated route guide line and the unnecessary vertexlike a duplicate spot are removed, the vertexes for maintaining astraight line of the route guide line the user's vehicle are added tothe user's vehicle front section , a step wherein the vertexes for acurve formation are added to the curve section of the route guide line,and a step wherein a route guide line to which the running trace radiusof the actually running vehicle is reflected, is generated using theadded vertexes.

In addition, the electronic apparatus 100 may perform a variable 3Dprocess while changing the heights with respect to the corrected routeguide line based on the distance to the user's vehicle (S504). Indetail, the step S504 may include a step wherein a virtual route guideline is generated at both sides of the processed route guide line, astep wherein the height values are calculated for the height values ofthe vertexes contained in the processed route guide line to increase inproportion to the distance, and a step for generating, throughpolygonization, surfaces with respect to the vertexes of the route guideline the height value of which has been calculated, and the vertexesincluded in the virtual route guide lines, thus performing a 3D process.

In addition, the electronic apparatus 100 may generate a route guideindicator by mapping the texture having a displacement onto the 3D databased on the speed of the vehicle (S505).

In addition, the electronic apparatus 100 may output the route guideindicator on the screen through the AR (S506). Here, the output screenwill be described in detail with reference to FIG. 16.

FIG. 16 is a view illustrating a route guide screen according to anembodiment of the present invention. Referring to FIG. 16, theelectronic apparatus 100 according to an embodiment of the presentinvention may display together the route screen guide screen (leftscreen) on the AR and the route guide screen (right screen) on the map.

In this case, the AR provision unit 160 may generate an indicator whichis overlapped on the AR for a guide on the AR.

As an example, as illustrated in FIG. 16, the AR provision unit 160 maygenerate a route guide indicator 1601, a lane change guide indicator1602, and a line departure guide indicator 1603. In addition, the ARprovision unit 160 may output the generated indicators on the AR.

FIG. 17 is a view depicting an implementation form wherein the cameraand the electronic apparatus are separate types according to anembodiment of the present invention. Referring to FIG. 17, a vehicleblack box 200 which is provided separate from the vehicle navigation 100may constitute the system according to an embodiment of the presentinvention using the wired/wireless communication method.

The vehicle navigation 100 may include, but not limited to, a displayunit 145 provided at a front portion of a navigation housing 191, anavigation controlling key 193, and a navigation microphone 195.

The vehicle black box 200 may obtain a data of a vehicle during therunning or stop of the vehicle. Namely, it is possible to photograph theimages during the running of the vehicle and the images even when thevehicle is stop. The quality of the images obtained through the vehicleblack box 200 may be constant or vary. As an example, the quality of theimages before or after the occurrence of an accident may be high, and inthe normal occasions, the quality of the images is low so as to minimizethe required storing space since it needs to store important images.

The vehicle black box 200 may include, but not limited to, a black boxcamera 222, a black box microphone 224 and an attaching unit 281.

Meanwhile, FIG. 17 depicts that the vehicle black box 200 providedseparate from the vehicle navigation 100 is connected in awired/wireless communication method, but the vehicle navigation 100 andthe vehicle black box 200 may not be connected in the wired/wirelesscommunication method. In this case, if a storing medium capable ofstoring the photographed images of the black box 200 is inserted in theelectronic apparatus 100, the electronic apparatus 100 may have afunction of the vehicle navigation 100 or the vehicle navigation 100 maybe integrally provided. This configuration, as depicted in FIG. 18, willbe described in detail.

FIG. 18 is a view depicting the implementation form wherein the cameraand the electronic apparatus are integral, according to an embodiment ofthe present invention. Referring to FIG. 18, if the electronic apparatushas a camera function, the user may install the electronic apparatuswhich allows a camera portion of the electronic apparatus to photographthe forward scenes of the vehicle and the display portion of theelectronic apparatus to recognize the user, thus implementing the systemaccording to an embodiment of the present invention.

FIG. 19 is a view depicting the implementation form by using the HUD(Head-UP Display) and the electronic apparatus according to anembodiment of the present invention. Referring to FIG. 19, theelectronic apparatus may display the AR guide screen on the head-updisplay with the help of the head-up display and the wired/wirelesscommunication.

Meanwhile, the control method of the electronic apparatus according tovarious embodiments of the present invention are implemented in the formof program codes, which may be provided to each server or device in astate where such program devices are stored in various non-transitorycomputer readable medium.

The non-transitory computer readable medium is not a medium, forexample, a register, a cash, a memory, etc., which is designed to storefor short time period, but a medium which may store datasemi-permanently and may be read by a device. In detail, the abovevarious applications or programs may be stored in the non-transitoryreadable medium, for example, CD, DVD, a hard disk, a blue ray disk,USB, a memory card, ROM, etc.

As the present invention may be embodied in several forms withoutdeparting from the spirit or essential characteristics thereof, itshould also be understood that the above-described examples are notlimited by any of the details of the foregoing description, unlessotherwise specified, but rather should be construed broadly within itsspirit and scope as defined in the appended claims, and therefore allchanges and modifications that fall within the meets and bounds of theclaims, or equivalences of such meets and bounds are therefore intendedto be embraced by the appended claims.

What is claimed is:
 1. A control method of an electronic apparatus,comprising: identifying a line region from a driving-related image dataof a vehicle; generating a line information corresponding to a lanewhere the vehicle is located from an image data of the identified lineregion portion; generating a position information of a lane where thevehicle is located, using at least one of the generated line informationand the lane information of a road where the vehicle is located; andperforming a driving-related guide of the vehicle using the generatedlane position information.
 2. The method of claim 1, wherein the lineinformation includes a line type information and a line colorinformation which each correspond to the lines formed at both sides ofthe lane where the vehicle is located.
 3. The method of claim 1, whereinthe generating the lane position information includes: obtaining a laneinformation of the road where the vehicle is located from a map data;determining whether or not the vehicle is located on the first lane orthe last lane of the road using the generated line information; andgenerating the position information of the lane where the vehicle islocated by using the lane information of the road, if the vehicle islocated on the first lane or the last lane.
 4. The method of claim 3,wherein the generating the lane position information further includes:wherein when the lane where the vehicle is located changes to the lanepositioning between the first lane and the last lane according to thelane change of the vehicle, updating the generated lane positioninformation with the changed lane position information.
 5. The method ofclaim 4, wherein when the lane where the vehicle is located changes fromthe lane which is located between the first or last lane to the firstlane or the last lane according to the lane change of the vehicle,regenerating the position information of the lane where the vehicle islocated by using the lane information of the road where the vehicle islocated.
 6. The method of claim 1, wherein the performing thedriving-related guide of the vehicle includes: outputting a lane changeguide using the navigation route of the vehicle and the lane positioninformation.
 7. The method of claim 1, wherein the performing thedriving-related guide of the vehicle includes: outputting a lane guidewhere the vehicle is located using the lane position information.
 8. Themethod of claim 2, further comprising: outputting an appropriate linedeparture guide based on the line type information which is identifiedbased on the line information.
 9. The method of claim 6, the outputtingcomprises: generating an indicator for performing the driving-relatedguide; and outputting the generated indicator through the augmentedreality.
 10. An electronic apparatus, comprising: a line informationgeneration unit which identifies a line region from a driving-relatedimage data of a vehicle and generates a line information correspondingto a lane where the vehicle is located from the image data of theidentified line region portion; a lane position information generationunit which generates a position information of a lane where the vehicleis located, using at least one of the generated line information and thelane information of the road where the vehicle is located; and a controlunit which performs a driving-related guide of the vehicle using thegenerated lane position information.
 11. The apparatus of claim 10,wherein the line information includes a line type information and a linecolor information which each correspond to each lines formed at bothsides of the lane where the vehicle is located.
 12. The apparatus ofclaim 10, wherein the lane position information generation unit isconfigured to obtain a lane information of the road where the vehicle islocated from a map data, determine, using the generated lineinformation, whether or not the vehicle is located on the first lane orthe last lane of the road, and if the vehicle is located on the firstlane or the last lane, generate a position information of the lane wherethe vehicle is located by using the lane information of the road. 13.The apparatus of claim 12, wherein when the lane where the vehicle islocated changes to the lane between the first lane and the last laneaccording to the lane change of the vehicle, the lane positioninformation generation unit updates the generated lane positioninformation with the changed lane position information.
 14. Theapparatus of claim 13, wherein when the lane where the vehicle islocated changes from the lane which is located between the first or lastlanes to the first lane or the last lane according to the lane change ofthe vehicle, the lane position information generation unit regenerates aposition information of a lane where the vehicle is located, by usingthe lane information of the road where the vehicle is located.
 15. Theapparatus of claim 10, wherein the control unit controls for the outputunit to output a lane change guide using the navigation route of thevehicle and the lane position information.
 16. The apparatus of claim10, wherein the control unit is configured to control the output unitfor output a lane guide where the vehicle is located using the laneposition information.
 17. The apparatus of claim 11, wherein the controlunit is configured to control the output unit to select and output anappropriate line departure guide based on the line type informationwhich is identified based on the line information.
 18. The apparatus ofclaim 15, wherein the control unit is configured to generate anindicator for performing the driving-related guide and control theoutput unit to output the generated indicator through the augmentedreality.
 19. A non-transitory computer readable storage mediumcontaining instructions, that when executed by one or more processors,cause the one or more processors to perform a method, the methodcomprising: identifying a line region from a driving-related image dataof a vehicle; generating a line information corresponding to a lanewhere the vehicle is located from an image data of the identified lineregion portion; generating a position information of a lane where thevehicle is located, using at least one of the generated line informationand the lane information of a road where the vehicle is located; andperforming a driving-related guide of the vehicle using the generatedlane position information.